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Abstract 

The rank minimization problem is to find the lowest-rank matrix in 
a given set. Nuclear norm minimization has been proposed as an convex 
relaxation of rank minimization. Recht, Fazel, and Parrilo have shown 
that nuclear norm minimization subject to an afhne constraint is equiv- 
alent to rank minimization under a certain condition given in terms of 
the rank-restricted isometry property. However, in the presence of mea- 
surement noise, or with only approximately low rank generative model, 
the appropriate constraint set is an ellipsoid rather than an afiine space. 
There exist polynomial-time algorithms to solve the nuclear norm mini- 
mization with an ellipsoidal constraint, but no performance guarantee has 
been shown for these algorithms. In this paper, we derive such an explicit 
performance guarantee, bounding the error in the approximate solution 
provided by nuclear norm minimization with an ellipsoidal constraint. 
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1 Introduction 



The rank minimization problem is to find the lowest rank matrix in a given set 
C IFHBOlj . i.e., 

min rank(X) 
subject to X E C. 

In particular, there are applications such as matrix completion and minimum 
order system identification [J that require the reconstruction of low-rank matrix 
X G C™^" from the linear measurement b ~ AX e C obtained with a given 
linear operator A : C™^" ^ C^. In this case, the set C is given as an afhne 
space by C = {X : AX = b} and we are solving an inverse problem AX = b for 
X with the a priori information that the true solution is a low-rank matrix. 

In general, rank minimization is a difficult non-convex optimization problem 
and no polynomial time algorithm has been proposed to date. Nuclear norm 
minimization [FHBOlj is a convex relaxation of the rank minimization problem 
with a convex set C. Recht, Fazel, and Parrilo derived a performance guarantee 
for nuclear norm minimization with an afhne constraint |RFP07| . A sufficient 
condition for the performance guarantee is given in terms of the rank-restricted 
isometry property of the linear operator A. Roughly, when A is nearly an 
isometry for low-rank matrices, rank minimization is equivalent to nuclear norm 
minimization and hence can be solved in polynomial time. 

However, in some applications of rank minimization such as minimum order 
system approximation, reduced order controller design, and the Euclidean dis- 
tance matrix problem [LLR95| . the inverse problem AX ~ b with given linear 
operator A and measurement b may not admit a low-rank solution. For example, 
in minimum order system approximation, the given system cannot be described 



^For more applications of rank minimization with an affine constraint, see [RFP07) . [Faz02) . 
and the reference therein. 
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by a low-rank matrix but can be well approximated by one. In this case, the min- 
imum rank of solutions to AX ~ b, which is given by minx{rank(X) : AX ~ b}, 
can be higher than the desired target rank. Another possibility is that there is 
additive noise in the measurements. Again the inverse problem AX = b may 
not admit a low-rank solution. Instead, in order to find a low-rank approximate 
solution whose rank is lower than the target value required by the application, 
the set C can be modified to an ellipsoid given by 

C = {X : WAX -b\\^^e}. (2) 

The resulting rank minimization problem defined by ([T]) and ^ is hard, 
and its nuclear norm convex relaxation can be used to obtain approximate solu- 
tions. In fact, there exist polynomial-time algorithms to solve the nuclear norm 
minimization problem with an ellipsoidal constraint (e.g. jFHBOlj . [CCSOSp . 
However, while empirically effective, a theoretical performance guarantee for 
those algorithms has been missing. Our goal in this paper is to close this gap in 
theory. We arc motivated by the analogy established by Recht, Fazel, and Par- 
rilo [RFP07j between the rank minimization problem and £q norm minimization, 
or equivalently compressed sensing, for the affine constraint case. This analogy 
extends to the convex relaxations of these problems, nuclear norm minimization 
and £i norm minimization, respectively |RFP07] . 

For the affine constraint case, Candes and Tao [CT05j have given a sufficient 
condition for the equivalence of £o norm minimization to its £i relaxation (or ba- 
sis pursuit) in the sense that both problems admit the same and unique solution. 
The condition is given in terms of the sparsity-restrictcd isometry property of 
the sensing matrix. For the ellipsoidal constraint case, also known as the noisy 
and compressible signal case, Candes extended the performance guarantee of £i 
norm minimization showing that the error in the sparse approximate solution is 
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bounded by a weighted sum of the best sparse approximation error of the true 
sohition and a bound on the energy of the noise in the measurement |Can08] . 
An analogous performance guarantee for nuclear norm minimization with an 
ellipsoidal constraint has not been available to date. 

In this paper, we seek the relation between the rank minimization problem 
with an ellipsoidal constraint and its convex relaxation. Basically, we use an 
analogue of the approach by Candes for £i norm minimization [CanOSj . The 
extended performance guarantee is given in terms of the rank-restricted isom- 
etry property and bounds the error in the low-rank approximate solution by 
a weighted sum of the error in the best low-rank approximation of the true 
solution, and a bound on the energy of the measurement noise. 

2 Performance Guarantee 

Consider two Hilbert spaces C™'^" and C^. For X,Y e C'"^", the inner prod- 
uct is defined by {X,Y)£mxn = Tr{Y^ X), where denotes the Hermitian 
transpose of Y. Then the induced Hilbert-Schmidt norm on C™^" is the Frobe- 
nius norm and will be denoted by \\-\\p. For x,y G C^, the inner product is 
defined by {x,y)cp = y^x. Then the induced Hilbert-Schmidt norm of C is 
the Euclidean norm and will be denoted by jl-jlj- 

The setting for the low-rank matrix recovery and approximation problem 
is the following. The measurement (with perturbation) of an unknown matrix 
X g C™^" is given as b = AX + u with ||i^||2 e where A : C™^" ^ is 
a given linear operator. The inverse problem is to recover matrix X, which is 
considered as an unknown true solution, with the side information that X has 
low rank or can can be well-approximated by such a matrix. Accordingly, the 
problem may be formulated as in (jT]), with the ellipsoidal constraint The 
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convex relaxation of this problem is the nuclear norm minimization problem 

min 1 1 X j 

p. XGC™x" 

subject to \\AX — bW^ ^ e, 

where \\X\\^ denotes the nuclear norm, which is the sum of the singular values 
of X. Problem P admits a low-rank solution that is a low-rank approximate 
solution to the original inverse problem. The quality of this approximate solu- 
tion can be guaranteed subject to a condition on the rank-restricted isometry 
constant. 

Given a linear operator A : C'"^" C, the rank- restricted isometry con- 
stant Sr{A) is defined as the minimum constant that satisfies 

(1 - 6r{A)) \\X\\l ^ \\AX\\l < (1 + 6r{A}) \\X\\l , (3) 

for all X e C™^" with rank(X) < r. ^ 

Theorem 2.1 Let X* be the solution to P. If A has the rank-restricted isometry 
constant 63r{A) < 1/(1 + 4/V3), then 

\\X* - X\\p < Ko \\X - Xr\\p + Kie, (4) 

where X^ denotes the best rank-r approximation of X given by 

Xr = arg^min {\\X - Z\\p : rank(Z) =^ r}. 

^ The definition of the rank-restricted isometry property is shghtly diff'crent from that in 
|RFP07| in the sense that the norms in the inequality arc squared in our definition. This 
is done for the consistency with the sparsity-rcstricted isometry for £o norm minimization 
|Can08| . 
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The constants Kq and Ki are given as 



V3 ; l-{l + i/V3)S3r{A) 




V3 J l-(l+4/V3)<53r(^)' 



respectively. 

The two terms \\X — Xr\\p and e in the bound of ^ reflect the compress- 
ibihty of matrix X , and the strength of the measurement noise, respectively. In 
general X may not be exactly low-rank with rank(X) ^ r but X admits a good 
low-rank approximation with small — X^H^. The measurements are also 
subject to a perturbation. These imperfections cause an error in the low-rank 
approximate solution obtained by the nuclear norm minimization. However, the 
gain of each term is explicitly bounded by a constant determined by the rank- 
restricted isometry constant. In particular, when e = and — X^H^ = 0, 
the solution obtained by the nuclear norm minimization coincides with the true 
solution X. Furthermore, an immediate corollary of Theorem 12.11 states that 
rank minimization with an ellipsoidal constraint can be solved by nuclear norm 
minimization in polynomial time in the sense that the distance between the solu- 
tions obtained by rank minimization and nuclear norm minimization is bounded 
as a linear function of \\X — Xr\\p and e. 

3 Proof of Performance Guarantee 

We first note the rank-restricted orthogonality property that follows from the 
rank-restricted isometry property. The following Proposition is an extension of 
Lemma 2.1 in |Can08| for the vector case to the matrix case. 

Definition 3.1 Given a set 5* = {-01, . . . , C C'"^", define a linear oper- 
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ator Lvf : C'*! ^ C"^" by 

L^a^^afe^Afe, VaeCl*!. (5) 
fe=i 

It follows from ^ that the adjoint operator L*^ : C'"^" — > C'*' is given by 

(L;X), = (X,^fc)c™x„, VA; = l,...,|vl;|, VXe (6) 

Note that for A : C'"^" ^ the operator composition AL^ : C'*! ^ 
admits a matrix representation. Its pseudo-inverse is denoted by [^L^,]^. 

Remark 3.2 If the elements in ^! are pairwise orthogonal and normalized, then 
Lq, is an isometry and the projection P^r onto span(\I') is given by Pq, = L<i,L\,. 
1} ^ is a set o/ rank-one matrices, then rank(L$Q;) ^ \'^\ for all a G C'*'. 

Proposition 3.3 Suppose that linear operator A : C™^" C has the rank- 
restricted isometry constant 5r{A). For X <E C™'^", let X ~ 'Y^^j^i^'^'^ i^ji^j 
denote the singular value decomposition of X where ipj is a rank-one unit-norm 
matrix obtained by the outer product of left and right singular vectors corre- 
sponding to the j-th singular value Uj for j = 1, . . . ,rank(X). Similarly, for 
Y G C™^", let Y — '^^j^^^'^^ '^j^-'j denote the singular value decomposition of 
Y . If {ipj, ?/'fe)c™><" = f'^^ aZ/ j = 1, . . . , rank(X) and k = 1, . . . , rank(y) and 
rank(X) + rank(y) ^ r, then 

\{AX,AY)c.\^Sr{A)\\X\\p\\Yy. (7) 



Proof Let * = H'jjfli \ = {i^j}T=i ' and * = *U*'. Then L*, L 
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and are isometries. Therefore, together with the R-RIP of A, it follows that 

1 - SriA) ^ a„,i„(i|^ML$) ^ a„,ax(i|X^L5) ^ 1 + SriA). 

Note that L'^,A*AL<s, is an off-diagonal submatrix of L*~A*AL^, and there- 
fore also of L'~A*AL^ — Id, where Id is the identity matrix of compatible size. 
Hence 

{Ll,A*AL^) s; {LlA*AL-^ - Id) 

s$ ma.x{{l + Sr{A))-l,l-{l-Sr{A))} 

= 5r{A). 

Noting that X = Pq,X = L^L'^X and Y = P^s'Y = L^,L*^,Y, it follows 

\{AX,AY)cp\ = \{AL^LlX,AL^>Ll,Y)cp\ 

= \{[L'^iA* AL-q;]L'^X , L'^iY)0<s,n\ 

^ 0'max(i*'^*-4L^) ||L^Xj|2 ||L*'F||2 

^ Sr{A)\\LlX\\^\\Ll,Y\\, 

= Sr{A)\\L^LlX\\p\\L^,L*^,Y\\p 

= <5.(^)||x||^liy||^. 



Next we note certain properties of the nuclear norm. 

Lemma 3.4 LetX,reC™^". Then \\X + Y\\,^ = \\Xl^ + \\Y\\,^ if and only if 
X and Y are simultaneously diagonalizable into nonnegative matrices. 

Proof Let and r„ denote the sets of unitary matrices in C'"^™ and C"^", 
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respectively. By the variational principle, 



\XL = max Tt(U"XV). (8) 



Moreover, {U, V) is a maximizer of ([5]) if and only if U^XV is a diagonal matrix 
where the diagonal entries are singular values of X. Equation ([8|) implies 



\X + YL = max Tt(U"(X + Y)V) (9) 

c/Gr,„,yer„ ^ ' 

< max Tt(U"XV) + max Tt(U"YV) (10) 

= 11X11,, + liriL. 



Let {Uo,Vo) denotes a maximizer of ([9]). The equality in (|10p holds if and only 
if both U^XVa and {/(f^FVb are diagonal matrices and the diagonal entries of 
U^XVa and U^YVq correspond to the singular values of X and Y, respectively. 
Noting that the singular values are nonnegative completes the proof. | 

Corollary 3.5 (Lemma 2.3 in [RFP07]) Let X,Y e C"''". If XY" = and 
X"Y = 0, then 

\\X + Y\\=\\X\\+\\Y\\.... 



Proof Let X ~ 1]\Yj\V{^ and Y = [/2S2V2 denote the singular value decom- 
positions of X and y, respectively. The assumption implies that V^V2 — 
and U^U2 = 0. Let U ^\\J^ f/j ] and V = \Vx V2 ]■ By concatenating 
orthonomal columns to U and V ^ we construct unitary matrices U and Y which 
have I] and Y as their submatrices, respectively. Then (J7, Y^ simultaneously 
diagonalize X and Y into nonnegative matrices and hence the result follows by 
Lemma 13.41 | 
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Lemma 3.6 Let X G C'"^'", and suppose rank(X) r. The 

IIXIL s$ 11X11 <ri/2|lX|L. 



Proof Let [cfkYk^i denotes the singular values of X in decreasing order where 
r is the rank of X. Since cr^ ^ for all fc = 1, . . . , r, 



fc=l 



fc=i 



where the second inequality follows from the Cauchy-Schwartz inequality. Not- 
ing \\X\\p = Y.k=i ai^d ll^ll* = SLi completes the proof. | 

Proof of Theorem [2^1 

Let X = UYy^ denote the full singular value decomposition of X, where 
U e C™^™, E e C'"^", y e C"^". Let Uk, Vk denote the fc-th column of U and 
y, respectively. Then define four projection operators in terms of the Uk's and 
Vks: 



PiZ 



PiZ = 



PaZ 



J = l fc=l 



m r 



j=r+l k=l 



J2 E (^'"^^fc"") 

j = l fc=r+l 



m 71. 



E E {Z,Ujv")c'^x^Ujv". 

j=r+l k=r+l 



Obviously, Pi + P2 + P3 + Pa = I, where / is the identity operator on C"'^". 
Also, Xr = PiX. By construction, {PiZ){PiZ)" = and {Pi Z)" {PiZ) = 
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for all Z e C™^". Then Corollary [S3] implies 

\\{Pi+Pi)z\\^^ = \\PiZ\l + \\PiZ\l vzeC"^". (11) 

Also note that rank(PfeZ) ^ r for aU Z e C™^" and for fc = 1, 2, 3. 

Let E = X* — X and let P^E = J2j>i^j'^j^f ^'^ ^^le singular value de- 
composition of P^E with singular values in decreasing order. Here aj = if 
i > rank^PiE). For fc ^ 1, define projection operator Qk by 

kr 

QkZ ^ {Z,Ujvf)c^x^Ujvf . 

j={k-l)r+l 

Then we have P4E — X^fc^i QkE and rank(Qfe£') ^ r for all k ^ 1. Now, for 
all fc ^ 2, we have 

\\QkE\\p ^ \\QkE\\^ < \\Q,^_,E\l 

and therefore 
It follows that 

11^4^^ - QiE\\p - II E Qfc^^!!^^ ^ E \\QkE\\p ^ iip^^ii^ . (13) 

k^2 k>2 
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Next, since X* is a solution to P, 



^ \\X*t = \\X + Et, = UP,+P2 + Ps + P4){X + E)\\, 

^ II (Pi +P,){X + E)l- \\(P2 +Ps){X + E)\\^ 

^ \\PiiX + E)t + \\P^{X + E)\\^ - \\{P2 + P3)iX + P)||, 

^ llFlXjl^ - ||Pii?||^ + \\P,E\\^ - \\P^X\\^ - \\{P2+P3)X\\^ - \\(P2 + P3)E\\^ 



where the equahty in the third hne foUows from (|TT|) . Therefore 

\\P,E\l ^ \\P,Et + im + P3)Et + II^IL - ll^i^lL + 11(^2 + P3)Xt + \\PM 



(a) (b) 

Lemma 3.7 Let a > be a constant and let a;, ?/ G M satisfy + j/^ — 1. Then 

X + ay 

Proof Let (a;o,?/o) = arg max(^ j,) {cc + ay : + = 1}. We may assmne 



2^0 5 yo ^ and then y^ = \J\ — x\. Let /(x) = x+avT^a?. Then 



dx 



= 0. Therefore the maximum 2a/ ^/a"^ + 1 is achieved when 



Xo 



a/ Vcc^ + 1 and yo — 1/ Va^ + 1- | 

Define a constant 7 = We further bound (a) by 

\\PiE\i + m+P3)E\i 

^ r'^'\\P,E\\p + {2r)'/^\\{P2 + P3)E\\p 
^ ^r'/^\\{Pi+P2 + P3)E\\p, 

where the first inequahty foUows from Lemma 13.61 with rank(Pi£^) ^ r and 
rank((P2 + P3)E) ^ 2r and the second inequahty is obtained by invoking 
LemmaOwith x = ||Pi-B||^ / ||(Pi + P2 + P3)E\\p, y = \\{P2 + P3)E\\p / \\{Pi + P2 + P3)E\\p, 
and a ~ \/2. 
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Next we further bound (b) by 

\\Xt-\\P,Xt + \\{P2 + P3)X\l + \\P,Xt 

^ \\P,X\\^ + \\P4X\\^ + \\iP2 + P3)Xl - \\P,X\l + \m + P3)Xh + \\P4Xt 

^ 2\\{P2 + P3)X\l+2\\PiX\l 

s; 2(2r)i/2 1|(P2 + P3)X\\p + 2ri/2 1|P4^||^ 

s; 27ri/2 II (P2 + P3 + Pi)X\\F = ^ir'^^ \\X - XrWp . 

Therefore 

||P4£;|L < ^r'/'\m+P2+P3)E\\p + 2jr'/^\\X-Xr\\p 

^ 7ri/2 ||(Pi +P2+ P3)E + Qii;||^ + 27ri/2 ||x - Xr\\p . (14) 

Here we used the fact that by construction Qi is orthogonal to Pi, P2, and P3. 
Combining (|13|) and we have 

11^4^^ - Qi^ll^ < 7 ||(Pi + P2 + P-3)E + Qii;||^ + 27 11^ - XrWp . (15) 

Next we bound ||£; - {P^E - QiE)\\p = ||(Pi + P2 + P3)E + QiE\\p. Since 
rank((Pi + P2 + P3)E) ^ 2r and rank(Qi_E) ^ r, by the subadditivity of rank, 
rank(£' - {P4E - QiE)) 3r. 

Since P^E - QiE = Y.k^^ QkE 

\\A{E - {PiE - QiE))\\l = {A{E-{P4E-QiE)),AE)cP 

" V ' 

(c) 

- {A{E-{PiE-QiE)),AY,QkE)cp.{16) 

(d) 
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We bound (c) by 



{A{E - {P4E - QiE)) ,AE)cp 
^ \\A{E-iPiE-QiE))\\^\\AE\\^ 

^ 2eVl + <^3r(-4) 11^ - {PiE - QiE)\\p . (17) 

Here we used the rank-restricted isometry property of A with rank(i? — {P^E — 
QiE)) ^ 3r and the fact that 

\\AE\\^ = - X)||2 5$ ||6 - AX*\\^ + \\b- AX\\^ s$ 2e, 

because X* is a solution to P. Next we bound (d). For each fc ^ 2, 

\{A{E- {PiE - QiE)) , AQkE)cv \ 
= \{A{{Pi+P2 + P7,)E + QiE),AQkE)cA 

I {A{PiE + QiE),AQkE)cP \ + \ {A{P2 + P3)E, AQkE)cP \ 
^ SsriA) \\P,E + Q,E\\p WQkEWp + SsAA) \\ {P2 + Ps)^!!^ \\QkE\\p 
^ V263riA) II (Pi + P2 + P3)E + QiE\\p WQkEWp 

= V263riA)\\E-iPiE-QiE)\\p\\QkE\\p, (18) 

where the second inequahty foUows from Proposition 13.31 because QkPj = 0, 
QkQi ~ for j = 1,2,3 and fc ^ 2 and these projections are defined by pairwise 
orthogonal rank-one matrices. 



14 



Applying ^ and ^ to we have 
\\AiE-iP,E-Q^Eml 

\\E-{PiE-QiE)\\p he^TTS^A) + V26sr{A)Y,\\QkE\\ 
< 11^; - {PiE - QiE)\\p (2eVTTM^+ V2J3r(^)r-i/2 \\PiE\\^y^9) 

where the second inequaUty follows from (fT2|) . 

From the rank-restricted isometry property of A, 



\\A{E-{PiE~QiE))\\l ^ [\ - hr{A))\\E ~ (P^E - QxE)fp . (20) 



Combining (|19p and (pO)) . we obtain 

\\E - {P^E - Q^E)\\p ^ ae + pr-i/2||p_j^||^ 

where 



2^\ + hr{A) 
" 1 - <^3.(-4) 

^hr{A) 
^ \-hr{A\ 

Using (fTi|) and the fact that Qi is orthogonal to Pi, P2, and P3, 

||i;-(P4£^-giP)||^ ^ ae + 7pi|(Pi+P2+P3)i;||^ + 27p||X-X,||j, 

^ ae + 7/9 II (Pi + P2 + P3)£ + gi^ll^ + 27P ||X - X,||^ 
= ae + 7/9 IIP - (P4P - QiP)||^ + 27P ||X - X,||^ . 



To proceed we use the assumption (53r(-4) < which implies 1 — 7p > 
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0, hence 



11^; - {P4E 



QiE)\\p ^ il-jp)-'{ae + 2^p\\X-Xr\\F). 



Finally, 




11^ - {P^E - QiE)\\p + \\P^E - Q^E\\p 



(1 + 7) ll-B - (P^E - QiE)\\p + 2j\\X- Xr\ 



(1 - 7p)-i [(1 + 7)ae + 27(1 + p)\\X- Xr\\p] , 



where the second inequality follows from (fT5)l . | 

4 Conclusion 

In this paper, we derived an extended performance guarantee of nuclear norm 
minimization with an ellipsoidal constraint. Unlike existing performance guar- 
antee, this constraint accommodates problem formulation in which the ma- 
trix is only approximately low rank, or in which there is noise in the mea- 
surements. The condition for the performance guarantee is given in terms of 
the rank-restricted isometry property of the linear operator in the constraint. 
The new performance guarantee in this paper ensures the quality of a low-rank 
approximate solution obtained by nuclear norm minimization with an ellip- 
soidal constraint. Such an approximate solution can be found by using existing 
polynomial-time algorithms . 
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